Decreased patient discharges on weekends part 3: what do the leaders tell us?

BACKGROUND: Emergency department (ED) crowding is a significant challenge to providing safe and quality care to patients. We know that hospital and ED crowding is exacerbated on Mondays because fewer in-patients are discharged on the weekend. We evaluated barriers and potential solutions to improve in-patient flow and diminished weekend discharges, in hopes of decreasing the severe ED crowding observed on Mondays. METHODS: In this observational study, we conducted interviews of (a) leaders at The Ottawa Hospital, a major academic health sciences centre (nursing, allied health, physicians), and (b) leaders of community facilities (long-term care and chronic hospital) that receive patients from the hospital, and (c) home care. Each interview was conducted individually and addressed perceived barriers to the discharge of hospital in-patients on weekends as well as potential solutions. An inductive thematic analysis was conducted whereby themes were organized into a summary table of barriers and solutions. RESULTS: We interviewed 40 leaders including 30 nursing, physician, and allied health leaders from the hospital as well as 10 senior personnel from community facilities and home care. Many barriers to weekend discharges were identified, highlighting that this problem is complex with many interdependent internal and external factors. Fortunately, many specific potential solutions were suggested, in immediate, short-term and long-term time horizons. While many solutions require additional resources, others require a culture change whereby hospital and community stakeholders recognize that services must be provided consistently, seven days a week. INTERPRETATION: We have identified the complex and interdependent barriers to weekend discharges of in-patients. There are numerous specific opportunities for hospital staff and services, physicians, and community facilities to provide the same patient care on weekends as on weekdays. This will lead to improved patient flow and safety, and to decreased ED crowding on Mondays.

ABSTRAIT: CONTEXTE: Le surpeuplement des services d'urgence (SU) est un défi important pour fournir des soins sécuritaires et de qualité aux patients. Nous savons que le surpeuplement des hôpitaux et des urgences est exacerbé le lundi parce que moins de patients hospitalisés reçoivent leur congé le week-end. Nous avons évalué les obstacles et les solutions potentielles pour améliorer le flux de patients hospitalisés et diminuer les congés de fin de semaine, dans l'espoir de réduire le surpeuplement sévère observé le lundi. MéTHODES: Dans cette étude observationnelle, nous avons interviewé (a) des dirigeants de l'Hôpital d'Ottawa, un important centre universitaire des sciences de la santé (soins infirmiers, soins paramédicaux, médecins), et (b) des dirigeants d'établissements communautaires (soins de longue durée et hôpitaux de soins chroniques) qui reçoivent des patients de l'hôpital et (c) des soins à domicile. Chaque entrevue a été menée individuellement et a abordé les obstacles perçus au congé des patients hospitalisés le week-end ainsi que les solutions potentielles. Une analyse thématique inductive a été menée, dans le cadre de laquelle les thèmes ont été organisés en un tableau récapitulatif des obstacles et des solutions RéSULTATS: Nous avons interviewé 40 dirigeants, dont 30 chefs de file des soins infirmiers, des médecins et des professions paramédicales de l'hôpital, ainsi que 10 cadres supérieurs d'établissements communautaires et de soins à domicile. De nombreux obstacles aux congés de fin de semaine ont été cernés, ce qui souligne que ce problème est complexe et qu'il comporte de nombreux facteurs internes et externes interdépendants. Heureusement, de nombreuses solutions potentielles spécifiques ont été proposées, à court terme et à long terme. Bien que de nombreuses solutions exigent des ressources supplémentaires, d'autres exigent un changement de culture par lequel les intervenants hospitaliers et communautaires reconnaissent que les services doivent être fournis de façon uniforme, sept jours par semaine. INTERPRéTATION: Nous avons identifié les obstacles complexes et interdépendants aux sorties de fin de semaine des patients hospitalisés. Il existe de nombreuses possibilités précises pour le personnel et les services hospitaliers, les médecins et les établissements communautaires d'offrir les mêmes soins aux patients les fins de semaine que les jours de semaine. Cela permettra d'améliorer la circulation et la sécurité des patients, et de réduire le surpeuplement des urgences le lundi.

Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Comparison of peer-assisted learning with expert-led learning in medical school ultrasound education: a systematic review and meta-analysis.

INTRODUCTION: Teaching point-of-care ultrasonography (PoCUS) to medical students is resource intensive. Peer-assisted learning, where the teacher can be a medical student, may be a feasible alternative to expert-led learning. The objective of this systematic review and meta-analysis was to compare the PoCUS performance assessments of medical students receiving peer-assisted vs expert-led learning. METHODS: This study was submitted to PROSPERO (CRD42023383915) and reported with PRISMA guidelines. MEDLINE, Embase, ERIC, Education Source, Scopus, and Web of Science were searched from inception to November 2022. Inclusion criteria were studies comparing peer-assisted vs expert-led PoCUS teaching for undergraduate medical students. The primary outcome was performance assessment of PoCUS skills. Two reviewers independently screened citations and extracted data. The Cochrane risk-of-bias tool for randomized trials was used to assess study quality. Studies were included in the meta-analysis if mean performance assessment scores with standard deviations and sample sizes were available. A random-effects meta-analysis was conducted to estimate the accuracy score of practical knowledge test for each group. A meta-regression evaluated difference in mean scores. RESULTS: The search yielded 2890 citations; 1417 unique citations remained after removing duplicates. Nine randomized-controlled studies conducted in Germany, USA, and Israel, with 593 participants, were included in the meta-analysis. The included studies assessed teaching of abdominal, cardiac, thoracic, musculoskeletal, and ocular PoCUS skills. Most studies had some risk-of-bias concerns. The estimate accuracy score after weighting is 0.56 (95% CI [0.47, 0.65]) for peer-assisted learning and 0.59 (95% CI [0.49, 0.69]) for expert-led learning. The regression coefficient estimate is 0.0281 (95% CI [- 0.1121, 0.1683]); P value is 0.69. CONCLUSION: This meta-analysis found that peer-assisted learning was a reasonable alternative to expert-led learning for teaching PoCUS skills to medical students.

RéSUMé: INTRODUCTION: L'enseignement de l'échographie au point d'intervention (PoCUS) aux étudiants en médecine nécessite des ressources importantes. L'apprentissage assisté par les pairs, où l'enseignant peut être un étudiant en médecine, peut être une alternative possible à l'apprentissage dirigé par des experts. L'objectif de cette revue systématique et de cette méta-analyse était de comparer les évaluations de performance PoCUS d'étudiants en médecine bénéficiant d'un apprentissage assisté par des pairs par rapport à un apprentissage dirigé par des experts. MéTHODES: Cette étude a été soumise à PROSPERO (CRD42023383915) et rapportée selon les directives PRISMA. MEDLINE, Embase, ERIC, Education Source, Scopus et Web of Science ont été recherchés depuis leur création jusqu'en novembre 2022. Les critères d'inclusion étaient les études comparant l'enseignement du PoCUS assisté par des pairs à celui dirigé par des experts pour les étudiants en médecine de premier cycle. Le principal résultat était l'évaluation du rendement des compétences PoCUS. Deux évaluateurs ont indépendamment examiné les citations et extrait les données. L'outil Cochrane d'évaluation du risque de biais pour les essais randomisés a été utilisé pour évaluer la qualité des études. Les études ont été incluses dans la méta-analyse si les scores moyens d'évaluation des performances avec les écarts types et la taille des échantillons étaient disponibles. Une méta-analyse à effets aléatoires a été réalisée pour estimer le score de précision du test de connaissances pratiques pour chaque groupe. Une méta-régression a évalué la différence dans les scores moyens. RéSULTATS: La recherche a donné lieu à 2890 citations ; 1417 citations uniques ont été conservées après suppression des doublons. Neuf études contrôlées randomisées menées en Allemagne, aux États-Unis et en Israël, avec 593 participants, ont été incluses dans la méta-analyse. Les études incluses ont évalué l'enseignement des compétences PoCUS abdominales, cardiaques, thoraciques, musculo-squelettiques et oculaires. La plupart des études présentaient des risques de biais. Le score de précision estimé après pondération est de 0,56 (IC à 95 % : [0,47, 0,65]) pour l'apprentissage assisté par les pairs et de 0,59 (IC à 95 % : [0,49, 0,69]) pour l'apprentissage dirigé par des experts. L'estimation du coefficient de régression est de 0,0281 (IC à 95 % : [-0,1121, 0,1683]) ; la valeur P est de 0,69. CONCLUSION: Cette méta-analyse a montré que l'apprentissage assisté par les pairs était une alternative raisonnable à l'apprentissage dirigé par des experts pour enseigner les compétences PoCUS aux étudiants en médecine.

Implementing Competence Committees on a National Scale: Design and Lessons Learned.

Competence committees (CCs) are a recent innovation to improve assessment decision-making in health professions education. CCs enable a group of trained, dedicated educators to review a portfolio of observations about a learner's progress toward competence and make systematic assessment decisions. CCs are aligned with competency based medical education (CBME) and programmatic assessment. While there is an emerging literature on CCs, little has been published on their system-wide implementation. National-scale implementation of CCs is complex, owing to the culture change that underlies this shift in assessment paradigm and the logistics and skills needed to enable it. We present the Royal College of Physicians and Surgeons of Canada's experience implementing a national CC model, the challenges the Royal College faced, and some strategies to address them. With large scale CC implementation, managing the tension between standardization and flexibility is a fundamental issue that needs to be anticipated and addressed, with careful consideration of individual program needs, resources, and engagement of invested groups. If implementation is to take place in a wide variety of contexts, an approach that uses multiple engagement and communication strategies to allow for local adaptations is needed. Large-scale implementation of CCs, like any transformative initiative, does not occur at a single point but is an evolutionary process requiring both upfront resources and ongoing support. As such, it is important to consider embedding a plan for program evaluation at the outset. We hope these shared lessons will be of value to other educators who are considering a large-scale CBME CC implementation.

Evaluating Competence by Design as a Large System Change Initiative: Readiness, Fidelity, and Outcomes.

Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.

Design and Implementation of a National Program of Assessment Model - Integrating Entrustable Professional Activity Assessments in Canadian Specialist Postgraduate Medical Education.

Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country.

Coaching in Competence by Design: A New Model of Coaching in the Moment and Coaching Over Time to Support Large Scale Implementation.

Coaching is an increasingly popular means to provide individualized, learner-centered, developmental guidance to trainees in competency based medical education (CBME) curricula. Aligned with CBME's core components, coaching can assist in leveraging the full potential of this educational approach. With its focus on growth and improvement, coaching helps trainees develop clinical acumen and self-regulated learning skills. Developing a shared mental model for coaching in the medical education context is crucial to facilitate integration and subsequent evaluation of success. This paper describes the Royal College of Physicians and Surgeons of Canada's coaching model, one that is theory based, evidence informed, principle driven and iteratively and developed by a multidisciplinary team. The coaching model was specifically designed, fit for purpose to the postgraduate medical education (PGME) context and implemented as part of Competence by Design (CBD), a new competency based PGME program. This coaching model differentiates two coaching roles, which reflect different contexts in which postgraduate trainees learn and develop skills. Both roles are supported by the RX-OCR process: developing Relationship/Rapport, setting eXpectations, Observing, a Coaching conversation, and Recording/Reflecting. The CBD Coaching Model and its associated RX-OCR faculty development tool support the implementation of coaching in CBME. Coaching in the moment and coaching over time offer important mechanisms by which CBD brings value to trainees. For sustained change to occur and for learners and coaches to experience the model's intended benefits, ongoing professional development efforts are needed. Early post implementation reflections and lessons learned are provided.

Deidentifying Narrative Assessments to Facilitate Data Sharing in Medical Education.

PROBLEM: Narrative assessments are commonly incorporated into competency-based medical education programs. However, efforts to share competency-based medical education assessment data among programs to support the evaluation and improvement of assessment systems have been limited in part because of security concerns. Deidentifying assessment data mitigates these concerns, but deidentifying narrative assessments is time-consuming, resource intensive, and error prone. The authors developed and tested a tool to automate the deidentification of narrative assessments and facilitate their review. APPROACH: The authors met throughout 2021 and 2022 to iteratively design, test, and refine the deidentification algorithm and data review interface. Preliminary testing of the prototype deidentification algorithm was performed using narrative assessments from the University of Saskatchewan emergency medicine program. The algorithm's accuracy was assessed by the authors using the review interface designed for this purpose. Formal testing included 2 rounds of deidentification and review by members of the authorship team. Both the algorithm and data review interface were refined during the testing process. OUTCOMES: Authors from 3 institutions, including 3 emergency medicine programs, an anesthesia program, and a surgical program, participated in formal testing. In the final round of review, 99.4% of the narrative assessments were fully deidentified (names, nicknames, and pronouns removed). The results were comparable for each institution and specialty. The data review interface was improved with feedback obtained after each round of review and found to be intuitive. NEXT STEPS: This innovation has demonstrated viability evidence of an algorithmic approach to the deidentification of assessment narratives while reinforcing that a small number of errors are likely to persist. Future steps include the refinement of both the algorithm to improve its accuracy and the data review interface to support additional data set formats.

Only as Strong as the Weakest Link: Resident Perspectives on Entrustable Professional Activities and Their Impact on Learning.

Background Core to competency-based medical education (CBME) is the use of frequent low-stakes workplace-based assessments. In the Canadian context, these observations of performance are framed around entrustable professional activities (EPAs). Objective We aimed to explore residents' real-world perspectives of EPAs and their perceived impact on learning, because assessments perceived to be "inauthentic," or not truly reflective of their lived experiences, may interfere with learning. Methods Using constructivist grounded theory, we conducted 18 semistructured interviews in 2021 with residents from all programs that had implemented CBME at one tertiary care academic center in Canada. Participants were recruited via email through respective program administrators. Data collection and analysis occurred iteratively, and categories were identified using constant comparative analysis. Results Residents were strikingly polarized, perceiving EPAs as either a valuable opportunity for professional growth or as an onerous requirement that interfered with learning. Regardless of what view participants held, all perspectives were informed by: (1) the program administration and the perceived messaging from program to residents; (2) faculty assessors and their perceived degree of engagement, or "buy-in" with the EPA system; and ultimately (3) learner behavior. We theorized from these findings that all 3 aspects must be working in tandem for the assessment system to function as intended. Conclusions From the learners' perspective, there exists a dynamic, interdependent relationship between the 3 CBME stakeholders. As such, the perceived value of the EPA assessment system can only be as strong as the weakest link in the chain.

Implementation of the Canadian syncope pathway: a pilot non-randomized stepped wedge trial.

BACKGROUND: We developed the Canadian Syncope Pathway (CSP) based on the Canadian Syncope Risk Score (CSRS) to aid emergency department (ED) syncope management. This pilot implementation study assessed patient inclusion, length of transition period, as well as process measures (engagement, reach, adoption, and fidelity) to prepare for multicenter implementation. METHODS: A non-randomized stepped wedge trial at two hospitals was conducted over a 7-month period. After 2-3 months in the control condition, the hospitals crossed over in a stepwise fashion to the intervention condition. Study participants were ED and non-ED physicians, or their delegates, and patients (aged ≥ 18 years) with syncope. We aimed to analyze patient characteristics, ED management including disposition decision, and CSRS recommendations application for all eligible patients during the intervention period. Our targets were 95% inclusion rate, 70% adoption (proportion of physicians who applied the pathway), 60% reach (intervention applied to eligible patients) and 70% fidelity (appropriate recommendations application) for all eligible patients. Clinical Trials registration NCT04790058. RESULTS: 1002 eligible patients (mean age 56.6 years; 51.0% males) were included: 349 patients during the control and 653 patients during the intervention period. Physician engagement varied from 39.7% to 97.1% for presentation at meetings. Process measures for the first month and the end of the intervention were: adoption 70.7% (58/82) and 84.4% (103/122), reach 67.5% (108/160) and 55.0% (359/653), fidelity among patients with physician data form completion 86.3% (88/102) and 88.3% (294/333), versus fidelity among all eligible patients 83.8% (134/160) and 83.3% (544/653) respectively with no significant differences in fidelity at one month and the end of the intervention period. CONCLUSION: In this pilot study, we achieved all prespecified benchmarks for proceeding to the multicenter CSP implementation except reach. Our results indicate a 1-month transition period will be adequate though regular reminders will be needed during full-scale implementation.

RéSUMé: CONTEXTE: Nous avons mis au point la Canadian Syncope Pathway (CSP) basée sur le Canadian Syncope Risk Score (CSRS) pour aider les services d'urgence à gérer la syncope. Cette étude pilote de mise en œuvre a évalué l'inclusion des patients, la durée de la période de transition, ainsi que les mesures de processus (engagement, portée, adoption et fidélité) pour se préparer à la mise en œuvre multicentrique MéTHODES: Un essai par étapes non randomisé dans deux hôpitaux a été mené sur une période de 7 mois. Après 2 à 3 mois dans l'état de contrôle, les hôpitaux sont passés progressivement à l'état d'intervention. Les participants à l'étude étaient des médecins du service de l'urgence et non du service de l'urgence, ou leurs délégués, et des patients (âgés de 18 ans) atteints de syncope. Nous avons cherché à analyser les caractéristiques des patients, la prise en charge des urgences, y compris la décision de disposition, et l'application des recommandations du CSRS pour tous les patients admissibles pendant la période d'intervention. Nos cibles étaient le taux d'inclusion de 95 %, l'adoption de 70 % (proportion de médecins qui ont appliqué la voie), la portée de 60 % (intervention appliquée aux patients admissibles) et la fidélité de 70 % (application des recommandations appropriées) pour tous les patients admissibles. Enregistrement des essais cliniques NCT04790058. RéSULTATS: 1002 patients éligibles (âge moyen 56,6 ans; 51,0% d'hommes) ont été inclus : 349 patients pendant le contrôle et 653 patients pendant la période d'intervention. La participation des médecins variait de 39,7 % à 97,1 % pour la présentation aux réunions. Les mesures du processus pour le premier mois et la fin de l'intervention étaient les suivantes : adoption 70,7 % (58/82) et 84,4 % (103/122), atteinte de 67,5 % (108/160) et 55,0 % (359/653), fidélité chez les patients ayant rempli le formulaire de données médicales 86,3 % (88/102) et 88,3 % (294/333), versus fidélité chez tous les patients admissibles 83,8 % (134/160) et 83,3 % (544/653) respectivement, sans différence significative de fidélité à un mois et à la fin de la période d'intervention. CONCLUSION: Dans cette étude pilote, nous avons atteint tous les points de repère prédéterminés pour procéder à la mise en œuvre du PSC multicentrique, sauf la portée. Nos résultats indiquent qu'une période de transition d'un mois sera adéquate, bien que des rappels réguliers seront nécessaires pendant la mise en œuvre à grande échelle.

Point-of-Care-ultrasound in undergraduate medical education: a scoping review of assessment methods.

BACKGROUND: Point-of-Care-Ultrasound (POCUS) curricula have rapidly expanded in undergraduate medical education (UME). However, the assessments used in UME remain variable without national standards. This scoping review characterizes and categorizes current assessment methods using Miller's pyramid for skills, performance, and competence of POCUS in UME. A structured protocol was developed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR). A literature search of MEDLINE was performed from January 1, 2010, to June 15, 2021. Two independent reviewers screened all titles and abstracts for articles that met inclusion criteria. The authors included all POCUS UME publications in which POCUS-related knowledge, skills, or competence were taught and objectively assessed. Articles were excluded if there were no assessment methods used, if they exclusively used self-assessment of learned skills, were duplicate articles, or were summaries of other literature. Full text analysis and data extraction of included articles were performed by two independent reviewers. A consensus-based approach was used to categorize data and a thematic analysis was performed. RESULTS: A total of 643 articles were retrieved and 157 articles met inclusion criteria for full review. Most articles (n = 132; 84%) used technical skill assessments including objective structured clinical examinations (n = 27; 17%), and/or other technical skill-based formats including image acquisition (n = 107; 68%). Retention was assessed in n = 98 (62%) studies. One or more levels of Miller's pyramid were included in 72 (46%) articles. A total of four articles (2.5%) assessed for students' integration of the skill into medical decision making and daily practice. CONCLUSIONS: Our findings demonstrate a lack of clinical assessment in UME POCUS that focus on integration of skills in daily clinical practice of medical students corresponding to the highest level of Miller's Pyramid. There exists opportunities to develop and integrate assessment that evaluate higher level competencies of POCUS skills of medical students. A mixture of assessment methods that correspond to multiple levels of Miller's pyramid should be used to best assess POCUS competence in UME.

The influence of clinical coaching teams on quality of entrustable professional activity assessments.

Background: Coaching is an important component of workplace-based assessment in competency-based medical education. Longitudinal coaching relationships have been proposed to enhance the trainee-supervisor relationship and promote high-quality assessment. Objective: The objective of this study was to determine the influence of longitudinal coaching relationships on the quality of entrustable professional activity (EPA) assessments. Methods: EPAs (n = 174) completed by emergency medicine (EM) supervisors between July 2020 and June 2021 were extracted and divided into two groups; one group consisted of EPAs completed by supervisors when a longitudinal coaching relationship existed (n = 87) and the other group consisted of EPAs completed by the same supervisors when no coaching relationship existed (n = 87). Three physicians were recruited to rate the EPAs using the Quality of Assessment and Learning (QuAL) score, a previously published measure of EPA quality. An analysis of variance was performed to compare mean QuAL scores between the groups. Linear regression analysis was conducted to examine the relationship between trainee performance (EPA rating) and EPA assessment quality (QuAL score). Results: All raters completed the survey. The mean ± SD QuAL score in the coaching relationship group (3.63 ± 0.91) was higher than the no coaching relationship group (3.51 ± 1.10) but the difference was not statistically significant (p = 0.40). Supervisor was a significant predictor of QuAL score (p = 0.012) and supervisor alone accounted for 26% of the variability in QuAL scores (R2 = 0.26). There was no significant relationship between trainee performance and EPA assessment quality. Conclusions: The presence of a longitudinal coaching relationship did not influence the quality of EPA assessments.

Assessing the utility of a novel entrustment-supervision assessment tool.

BACKGROUND: Work-based assessments (WBAs) are increasingly used to inform decisions about trainee progression. Unfortunately, WBAs often fail to discriminate between trainees of differing abilities and have poor reliability. Entrustment-supervision scales may improve WBA performance, but there is a paucity of literature directly comparing them to traditional WBA tools. METHODS: The Ottawa Emergency Department Shift Observation Tool (O-EDShOT) is a previously published WBA tool employing an entrustment-supervision scale with strong validity evidence. This pre-/post-implementation study compares the performance of the O-EDShOT with that of a traditional WBA tool using norm-based anchors. All assessments completed in 12-month periods before and after implementing the O-EDShOT were collected, and generalisability analysis was conducted with year of training, trainees within year and forms within trainee as nested factors. Secondary analysis included assessor as a factor. RESULTS: A total of 3908 and 3679 assessments were completed by 99 and 116 assessors, for 152 and 138 trainees in the pre- and post-implementation phases respectively. The O-EDShOT generated a wider range of awarded scores than the traditional WBA, and mean scores increased more with increasing level of training (0.32 vs. 0.14 points per year, p = 0.01). A significantly greater proportion of overall score variability was attributable to trainees using the O-EDShOT (59%) compared with the traditional tool (21%, p < 0.001). Assessors contributed less to overall score variability for the O-EDShOT than for the traditional WBA (16% vs. 37%). Moreover, the O-EDShOT required fewer completed assessments than the traditional tool (27 vs. 51) for a reliability of 0.8. CONCLUSION: The O-EDShOT outperformed a traditional norm-referenced WBA in discriminating between trainees and required fewer assessments to generate a reliable estimate of trainee performance. More broadly, this study adds to the body of literature suggesting that entrustment-supervision scales generate more useful and reliable assessments in a variety of clinical settings.

Ready for practice? National recommendations for emergency medicine transition to practice curriculum.

BACKGROUND: Transition from residency to unsupervised practice represents a critical stage in learning and professional identity formation, yet there is a paucity of literature to inform residency curricula and emergency department transition programming for new faculty. OBJECTIVE: The objective of this study was to develop consensus-based recommendations to optimize the transition to practice phase of emergency medicine training. METHODS: A literature review and results of a survey of emergency medicine (EM) residency program directors informed focus groups of recent (within 5 years) EM graduates. Focus group transcripts were analyzed following conventional content analysis. Preliminary recommendations, based on identified themes, were drafted and presented at the 2022 Canadian Association of Emergency Physicians (CAEP) Academic Symposium on Education. Through a live presentation, symposium attendees representing the Canadian national EM community participated in a facilitated discussion of the recommendations. The authors incorporated this feedback to construct a final set of 14 recommendations, 8 targeted toward residency training programs and 6 specific to department leadership. CONCLUSION: The Canadian EM community used a structured process to develop 14 best practice recommendations to enhance the transition to practice phase of residency training as well as the transition period in the career of junior attending physicians.

ABSTRAIT: ARRIèRE-PLAN: La transition de la résidence à la pratique non supervisée représente une étape cruciale de l'apprentissage et de la formation de l'identité professionnelle, mais il y a peu de documentation pour éclairer les programmes de résidence et les programmes de transition des services d'urgence pour les nouveaux professeurs. OBJECTIF: L'objectif de cette étude était d'élaborer des recommandations consensuelles pour optimiser la transition vers la pratique de la formation en médecine d'urgence. MéTHODES: Une recension des écrits et les résultats d'un sondage auprès des directeurs des programmes de résidence en médecine d'urgence (GU) ont informé les groupes de discussion des diplômés récents (moins de cinq ans) en GU. Les transcriptions des groupes de discussion ont été analysées à la suite d'une analyse du contenu classique. Des recommandations préliminaires, fondées sur des thèmes déterminés, ont été rédigées et présentées au Symposium universitaire sur l'éducation de 2022 de l'Association canadienne des médecins d'urgence (ACMU). Au moyen d'une présentation en direct, les participants au symposium représentant la communauté nationale canadienne de la GU ont participé à une discussion dirigée sur les recommandations. Les auteurs ont intégré ces commentaires pour élaborer un ensemble final de 14 recommandations, 8 ciblant les programmes de formation en résidence et 6 ciblant le leadership ministériel. CONCLUSIONS: La communauté canadienne de la GU a utilisé un processus structuré pour élaborer 14 recommandations de pratiques exemplaires afin d'améliorer la transition à la phase de pratique de la formation en résidence ainsi que la période de transition dans la carrière des médecins traitants débutants.

Exploring gender influences in the quality of workplace-based assessments.

INTRODUCTION: Workplace-based assessments are an important tool for trainee feedback and as a means of reporting expert judgments of trainee competence in the workplace. However, the literature has demonstrated that gender bias can exist within these assessments. We aimed to determine whether gender differences in the quality of workplace-based assessment data exist in our residency training program. METHODS: This study was conducted at the University of Ottawa in the Department of Emergency Medicine. Four end-of-shift workplace-based assessments completed by men faculty and four completed by women faculty were randomly selected for each resident during the 2018-2019 academic year. Two blinded raters scored each workplace-based assessment using the Completed Clinical Evaluation Report Rating (CCERR), a published nine-item quantitative measure of workplace-based assessment quality. A 2 × 2 mixed measures analysis of variance (ANOVA) of resident gender and faculty gender was conducted, with mean CCERR score as the dependent variable. The ANOVA was repeated with mean workplace-based assessment rating as the dependent variable. RESULTS: A total of 363 workplace-based assessments were analyzed for 46 residents. There were no significant effects of faculty or resident gender on the quality of workplace-based assessments (p = 0.30). There was no difference in mean workplace-based assessment ratings between women and men residents (p = 0.92), and no interaction between resident and faculty gender (p = 0.62). Mean CCERR score was 25.8, SD = 4.2, indicating average quality assessments. CONCLUSIONS: We did not find faculty or resident gender differences in the quality of workplace-based assessments completed in our training program. While the literature has previously demonstrated gender bias in trainee assessments, our results are not surprising as assessment culture varies by institution and program. Our study cautions against generalizing gender bias across contexts, and offers an approach that educators can use to evaluate whether gender bias in the quality of trainee assessments exists within their program.

RéSUMé: INTRODUCTION: Les évaluations sur le lieu de travail constituent un outil important pour le retour d'information des stagiaires et comme moyen de rapporter les jugements d'experts sur les compétences des stagiaires sur le lieu de travail. Cependant, la littérature a démontré que des préjugés sexistes peuvent exister dans ces évaluations. Nous avons cherché à déterminer s'il existe des différences entre les sexes dans la qualité des données d'évaluation sur le lieu de travail dans notre programme de formation en résidence. MéTHODES: Cette étude a été menée à l'Université d'Ottawa dans le département de médecine d'urgence. Quatre évaluations en fin de poste de travail complétées par des professeurs hommes et 4 complétées par des professeurs femmes ont été sélectionnées au hasard pour chaque résident au cours de l'année universitaire 2018-2019. Deux évaluateurs en aveugle ont noté chaque évaluation sur le lieu de travail à l'aide du Completed Clinical Evaluation Report Rating (CCERR), une mesure quantitative publiée en neuf points de la qualité de l'évaluation sur le lieu de travail. Une analyse de variance (ANOVA) à mesures mixtes 2 × 2 du sexe des résidents et du sexe des enseignants a été réalisée, avec le score CCERR moyen comme variable dépendante. L'ANOVA a été répétée en prenant comme variable dépendante la note moyenne de l'évaluation sur le lieu de travail. RéSULTATS: Au total, 363 évaluations sur le lieu de travail ont été analysées pour 46 résidents. Il n'y avait aucun effet significatif du sexe du corps professoral ou du résident sur la qualité des évaluations en milieu de travail (p = 0,30). Il n'y avait pas de différence dans les évaluations moyennes sur le lieu de travail entre les femmes et les hommes résidents (p = 0,92), et pas d'interaction entre le sexe du résident et celui de la faculté (p = 0,62). Le score moyen du CCERR était de 25,8, SD = 4,2, ce qui indique des évaluations de qualité moyenne. CONCLUSIONS: Nous n'avons pas constaté de différences entre les sexes au sein du corps professoral ou des résidents en ce qui concerne la qualité des évaluations en milieu de travail effectuées dans le cadre de notre programme de formation. Bien que la littérature ait déjà démontré l'existence de préjugés sexistes dans les évaluations des stagiaires, nos résultats ne sont pas surprenants car la culture de l'évaluation varie selon les établissements et les programmes. Notre étude met en garde contre la généralisation des préjugés sexistes dans tous les contextes et propose une approche que les éducateurs peuvent utiliser pour évaluer s'il existe des préjugés sexistes dans la qualité des évaluations des stagiaires au sein de leur programme.

A systematic review of sufentanil for the management of adults with acute pain in the emergency department and pre-hospital setting.

BACKGROUND: Pain is commonly encountered in the Emergency Department (ED) and pre-hospital setting and often requires opioid analgesia. We sought to synthesize the available evidence on the effectiveness of sufentanil for acute pain relief for adult patients in the pre-hospital or ED setting. METHODS: This systematic review was conducted in accordance with PRISMA guidelines. Medline, Embase, Cochrane CENTRAL, and CINAHL were searched from inception to February 1, 2022. The grey literature was also searched. We included randomized controlled trials of adult patients with acute pain who were treated with sufentanil. Two reviewers independently completed screening, full text review, and data extraction. Primary outcome was reduction in pain. Secondary outcomes included adverse events, need for rescue analgesia, and patient and provider satisfaction. Risk of bias was assessed using the Cochrane Risk of Bias 2 tool. A meta-analysis was not performed due to heterogeneity. RESULTS: Of 1120 unique citations, four studies (3 ED and 1 pre-hospital) met full inclusion criteria (n = 467 participants). The overall quality of the included studies was high. Intranasal (IN) sufentanil was superior to placebo for pain relief at 30 min (difference 20.8%, 95% CI 4.0-36.2%, p = 0.01). Both IN (two studies) and IV sufentanil (one study) were comparable to IV morphine. Mild adverse events were common and there was a higher propensity for minor sedation in patients receiving sufentanil. There were no serious adverse events requiring advanced interventions. CONCLUSION: Sufentanil was comparable to IV morphine and was superior to placebo for rapid relief of acute pain in the ED setting. The safety profile of sufentanil is similar to IV morphine in this setting, with minimal concern for serious adverse events. The intranasal formulation may provide an alternative, rapid, non-parenteral route that could benefit our unique emergency department and pre-hospital patient population. Due to the overall small sample size of this review, larger studies are required to confirm safety.

How Emergency Medicine Residents Have Conversations About Life-Sustaining Treatments in Critical Illness: A Qualitative Study Using Inductive Thematic Analysis.

STUDY OBJECTIVE: The inherent pressures of high-acuity, critical illness in the emergency department create a unique environment whereby acute goals-of-care discussions must be had with patients or substitute decision makers to rapidly decide between divergent treatment paths. Among university-affiliated hospitals, resident physicians are often conducting these highly consequential discussions. This study aimed to use qualitative methods to explore how emergency medicine residents make recommendations regarding life-sustaining treatments during acute goals-of-care discussions in critical illness. METHODS: Using qualitative methods, semistructured interviews were conducted with a purposive sample of emergency medicine residents in Canada from August to December 2021. Inductive thematic analysis of the interview transcripts was conducted using line-by-line coding, and key themes were identified through comparative analysis. Data collection continued until thematic saturation was reached. RESULTS: Seventeen emergency medicine residents from 9 Canadian universities were interviewed. Two factors guided residents' treatment recommendations (a duty to provide a recommendation and the balance between disease prognosis and patient values). Three factors influenced residents' comfort when making recommendations (time constraints, uncertainty, and moral distress). CONCLUSION: While conducting acute goals-of-care discussions with critically ill patients or their substitute decision makers in the emergency department, residents felt a sense of duty to provide a recommendation informed by an intersection between the patient's disease prognosis and the patient's values. Their comfort in making these recommendations was limited by time constraints, uncertainty, and moral distress. These factors are important for informing future educational strategies.

Examining enablers and barriers to entrustable professional activity acquisition using the theoretical domains framework: A qualitative framework analysis study.

Background: Without a clear understanding of the factors contributing to the effective acquisition of high-quality entrustable professional activity (EPA) assessments, trainees, supervising faculty, and training programs may lack appropriate strategies for successful EPA implementation and utilization. The purpose of this study was to identify barriers and facilitators to acquiring high-quality EPA assessments in Canadian emergency medicine (EM) training programs. Methods: We conducted a qualitative framework analysis study utilizing the Theoretical Domains Framework (TDF). Semistructured interviews of EM resident and faculty participants underwent audio recording, deidentification, and line-by-line coding by two authors, being coded to extract themes and subthemes across the domains of the TDF. Results: From 14 interviews (eight faculty and six residents) we identified, within the 14 TDF domains, major themes and subthemes for barriers and facilitators to EPA acquisition for both faculty and residents. The two most cited domains (and their frequencies) among residents and faculty were environmental context and resources (56) and behavioral regulation (48). Example strategies to improving EPA acquisition include orienting residents to the competency-based medical education (CBME) paradigm, recalibrating expectations relating to "low ratings" on EPAs, engaging in continuous faculty development to ensure familiarity and fluency with EPAs, and implementing longitudinal coaching programs between residents and faculty to encourage repetitive longitudinal interactions and high-quality specific feedback. Conclusions: We identified key strategies to support residents, faculty, programs, and institutions in overcoming barriers and improving EPA assessment processes. This is an important step toward ensuring the successful implementation of CBME and the effective operationalization of EPAs within EM training programs.

Stress and conflict from tacit culture forges professional identity in newly graduated independent physicians.

To transition successfully into independent practice, newly graduated independent physicians (new "attendings") undergo a process of professional identity formation (PIF) as a clinician within a new community of practice (CoP). PIF is crafted by socialization within a CoP including transfer of tacit knowledge. While certain tacit knowledge is critical for professional identity, we understand little how it shapes PIF. We set out to describe the tacit knowledge acquired by new attendings within a CoP and how it contributes to PIF. Informed by constructivist grounded theory, we interviewed 23 new attendings about the tacit knowledge they acquired in early practice. Data collection and analysis occurred iteratively. We identified themes using constant comparative analysis and generated a theory that underwent member checking and feedback. Implicit standards from group culture imparted high expectations on new attendings and led to internal stress. New attendings also encountered a tacit code of conduct as behavioral elements of group culture. These elements created external conflict between new attendings and group members such as departmental colleagues, consulting physicians, and other health professionals. Depending on the support they received, new attendings responded to the stress and conflict in three ways: they doubted, adjusted, or avoided. These strategies molded their professional identity, and moved them towards or away from the CoP as they navigated their transition and PIF. We describe a novel theory of how tacit group culture shaped new attending physicians' professional identity in a new community of practice. Internal stress and external conflict occurred due to high expectations and tacit culture elements. New attendings' doubt, adjust, or avoid responses, shaped by support they received, in turn crafted their professional identity. Education leaders should prepare graduating trainees to navigate aspects of transition to independent practice successfully.